Combining Transformer Embeddings with Linguistic Features for Complex Word Identification

Identifying which words present in a text may be difficult to understand by common readers is well-known subtask complexity analysis. The advent of deep language models has also established the new state-of-the-art this task means end-to-end semi-supervised (pre-trained) and downstream training of, mainly, transformer-based neural networks. Nevertheless, usefulness traditional linguistic features combination with encodings worth exploring, as computational cost needed for running such networks becoming more relevant energy-saving constraints. This study explores lexical prediction (LCP) combining pre-trained adjusted transformer different types features. We apply these over classical machine learning classifiers. Our best results are obtained applying Support Vector Machines on an English corpus LCP solved regression problem. show that can useful tasks improve performance systems.